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Abstract 

In this study, we introduce a novel variant and application of the Collabo¬ 
rative Representation based Classihcation in spectral domain for recognition 
of the hand gestures using the raw surface Electromyography signals. The 
intuitive use of spectral features are explained via circulant matrices. The 
proposed Spectral Collaborative Representation based Classification (SCRC) 
is able to recognize gestures with higher levels of accuracy for a fairly rich 
gesture set. The worst recognition result which is the best in the literature 
is obtained as 97.3% among the four sets of the experiments for each hand 
gestures. The recognition results are reported with a substantial number of 
experiments and labeling computation. 
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1. Introduction 

Electromyographic (EMG) signal based applications beyond clinical and 
rehabilitation purposes have been extensively studied not only to analyze and 
improve the sport mia, occupational [3j and art performance |1] in which 
kinesiological EMG signals are captured via surface EMG electrodes but also 
to create intuitive Human Machine Interfaces (HCIs). 

In the latter group, the control of exoskeletons, robotic prosthetic arms 
and hands are the high end applications that require classihcation of the EMG 
signal patterns acquired from the related muscles groups non-invasively. A 
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vast variety of classification methods have been proposed in the literature. 
Among the suggested classihcation schemes, the heuristic methods such as 
the variants of artihcial neural networks and fuzzy classihers [3 El Eli prevail 
among the related studies. Depending on the number of signal patterns to be 
classihed using different kind of features, methods for pre-processing, source 
separation and hltering, the accuracy of the classihcation methods range 
between 88-99.75% PHO]. 

The EMG signal measured on skin surface is not a mere signal but rather 
the superposition of the Motor Unit Action Potential (MUAP) of the many 
tiny muscle hbers mm- Due to the stochastic nature of MUAP, the EMG 
signal pattern might exhibit intra and interpersonal variations for the same 
muscle activity that makes the EMG pattern classihcation an intricate task 
for creating robust classihers. 

In concert with the advancements in pervasive computing technologies, 
the wearable gadgets in which high grade sensors embedded have been re¬ 
leased to the market. Thalmic Labs’ MYO armband [TB] is one of the recent 
technology which consists of eight surface EMG sensors along with the in¬ 
ertial measurement units with an ahordable price launched to the end user 
and the mobile application developers. Inevitably, accessibility of such kind 
of device which has been used for years in the research and laboratory envi¬ 
ronments garnered public attention. 

In this article, we introduce a signal pattern classihcation method in spec¬ 
tral domain and training procedures to classify the EMG signals obtained by 
8-channel MYO armband real-time on a continuous manner for hand gesture 
and posture recognition. The classihcation method is based on the Gollabo- 
rative Representation based Glassihcation (GRG) [H] which competes with 
the Sparse Representation based Glassihcation (SRG) [T5] with the same ac¬ 
curacy levels but much faster computation times. The drawbacks of the GRG 
compared to the SRG is that the GRG requires the observed and training sig¬ 
nal patterns in the dictionary to be of equal length. This restricts the use of 
GRG methods for gesture and posture recognition study in which the dura¬ 
tion of the gestures might vary for each repetition. On the other hand, the 
SRG methods promotes the representation hdelity whereas the GRG data 
hdelity which might give rise to low classihcation accuracy, if the data do 
not exhibit hdelity [16]. As the EMG signals are the products of complex 
stochastic processes, the GRG method yields poor recognition accuracy as 
we detail comparisons in the proceeding sections. 

The Spectral Gollaborative Representation based Glassihcation (SGRG) 
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proposed in this paper, overcomes these drawbacks and yield high recognition 
accuracy for a fairly rich hand gesture set. As the observed and training 
signal patterns must have equal lengths, a special training scheme is adapted 
which allows building a training dictionary with the representative columns 
for all possible observed signal patterns. Therefore, the need to spotting 
signal patterns on a streaming signal is automatically eliminated, and the 
boundary of the representative columns are implicitly coded in the training 
matrix. 

The contribution of this study to the literature are; 

• The spectral content; complex conjugate eigenvalue pairs are obtained 
from ID vectors by converting the observed signal pattern to a trajec¬ 
tory or a circulant matrix to capture the representative features of the 
EMG signals. 

• The gesture and posture recognition is performed in a continuous man¬ 
ner eliminating the requirements for spotting or picking the signal pat¬ 
terns on the streaming signals due to the proposed training scheme 
which implicitly embeds the signal boundaries on the representative 
columns. 

• The training phase is easy to implement thus the end user can obtain 
a training dictionary on the spot. The flexibility in building a training 
dictionary paves the way for the use of the procedures and methods 
introduced here to implement different applications, such as control of 
a bionic prosthesis. 

• The number of hand gestures are the highest among the similar studies. 
Our worst recognition result is over 97% with a substantial number of 
gesture labeling computation. 

The rest of the paper is organized as follows. In Section 2, a brief 
review of the CRC method is given then the circulant matrix approach 
is detailed. In Section 3, after giving the technical details of the MYO 
armband, training phase for the gestures and SCRC is eloborated. The 
experiment and simulation results are discussed in Section 4. The paper 
ends with conclusion and future works in Section 5. 
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2. Spectral Collaborative Representation Based Classification 

2.1. Collaborative Representation based Classification 

The spectral analysis which have been nsed to for decades in the stndies 
where random signal or stochastic processes are involved. Although random¬ 
ness in the systems might render prediction of some irregular features in the 
time domain, the spectral features such as eigenvalues and frequency might 
reveal valuable information of the underlying processes d- The fourier 
analysis is the widely used spectral analysis method for this purpose. In this 
study, as the EMG signals show randomness, we exploit the spectral analysis 
by employing the circulant matrix structure for eigenvalue decomposition. 

Both methods, the SRC and CRC address finding the linear representa¬ 
tion coefficients vector x from the linear systems of equation y = Ax where 
A = [Ai, A 2 ,..., An] G is the dictionary matrix in which the repre¬ 

sentatives from different classes are stacked as a column vector and y is the 
observed signal. In the objective function of the solution (Eq. different 
vector norms are utilized depending on the methods and requirements. 

h = argmin III/— Aa:||p-|-cT||a;||g (1) 

X 

In SRC solution ii regularization are used (p=2, q=l), where as the p 
and q becomes 1 or 2 in the CRC methods depending on the requirements 
such robustness of the classifier. When the objective function is solved by 
Regularized Least Square (RLS) in which £2 norm used for both terms of the 
objective function the solution turns out the ridge regression. We used the 
least square version of the CRC method which is called as CRC_RLS by the 
authors in [16] . The ridge regression solution of Eq. thus is obtained as 
X = Py where P = (A'^A + and a is the regularization parameter. 

Once the solution vector x is obtained the label of the observed signal is 
computed evaluating the minimum representation residuals given in Eq. 
where Si : M” —)■ MA is the selection operator that selects the coefficients of 

class while keeping other coefficients zero in the solution vector x. 

min rfiy) = \\y - A 6 i{x )\\2 (2) 

I 

The SRC methods yield good classification results as long as the dictio¬ 
nary matrix is overcomplete whereas the CRC do not require overcomplete 
dictionaries. In addition with the lower number of dictionary representatives. 
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the regression operator P is only computed and stored once and is coded over 
the anew observed patterns for the solution vector x which makes the classi- 
hcation method perform faster then the SRC in real time applications. 

Infact, even though new approaches have been introduced in the literature 
such as Block Sparse or Structured Sparse ii solvers [IHl US] and Block SRC 
methods perform much better then the plain SRC in terms of the computa¬ 
tional time without compromising the accuracy levels m, the computation 
times are not satisfactory on the mobile devices such as phones and tablets 
for classihcation due to the fact that all the solvers are based on iterative 
algorithms pT] , 


2.2. Circulant Matrices and Spectral Collaboration based Classification 

Circulant matrices which appear naturally in many systems of equa¬ 
tion such as that of dynamical systems, convolution and vibration theories 
HU 1221123]. A circulant matrix is constructed by its hrst row by shifting 
the elements of the row vector in clockwise or counterclockwise directions 
circularly. Depending of the direction of their diagonals (right or left circu- 
lants) they take the form of Toeplitz and Hankel matrices respectively which 
have wide range of application areas from compressing sensing, hlter design, 
singular spectrum analysis to inverse eigenvalue problems [2^ 123112^ 127] . 

Let assume a vector a = [uq, Ui,..., n„], the right circulant C = circ{a) G 

l^nxn PecQjYies; 
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Any circulant can be diagonalized by a unitary or Discrete Fourier Trans¬ 
formation (DFT) matrix due to the their cyclic structure. This factorization 
is also a similarity transformation. If any two matrices A and B are similar, 
one of the them can be expressed as B = T~^AT where T is an invertible 
matrix. If T is a DFT matrix Fh (Eq. |^and Eq. |^, the matrix A becomes 
a diagonal matrix, the entries of which are the eigenvalues of the circulant 
transformed. 


A = F^CFn 


(4) 


5 






In these equation diag{A) = {Ai, A 2 ,..., A„} and the matrix operator 
(o)^ is the Hermitian transpose. 


Fn = 
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(5) 


The columns of the DFT matrix with the entries W = exp thus 
becomes the eigenvectors of the circulant matrix. As all the eigenvectors 
of the decomposed circular matrices with the same size are all the same, 
the only discriminative subspace are only defined by the eigenvalues of the 
matrices. These subspaces which are shift invariant can be used as a dictio¬ 
nary matrix in the representation based classifications. The time complexity 
of the eigenvalue decomposition with a number of operation 0 {n\ogn) is 
lower then the conventional decomposition methods, therefore, eigenvalue 
decomposition of the circulants is faster then the conventional eigenvalue 
decomposition algorithms |25j . 

In the SCRC, we build a training dictionary using the eigenvalues of the 
observed signal as the features. Since, complex numbers are involved, the 
Hermitian transpose of the vectors is used in the inner product space. In the 
complex domain, the solution vector x is obtained by the equation x = PcX 
where; 


P^ = {A^A + aI)-^A^ ( 6 ) 

The training matrix is normalized and centered in the CRC methods while 
the observed signal y is only centered. It is important to note that since 
the spectral features are used in the Fourier domain, Hermitian conjugate 
transpose is used for all inner product operations. 


3. SCRC Application in EMC signal Classification 

3.1. MYO Armband and Gestures 

The MYO armband which has been recently released to the market by 
Thalmic Labs revolutionized the EMG based studies in which previously 
the EMG signals were only acquired in the laboratory or research environ¬ 
ment with the expensive hardware. The sensor armband has 8-channel EMG 
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sensor group along supported with an Inertial Measurement Unit (IMU) 
which reports linear and rotational acceleration as well as the rotation angles 
around 3-axes. The sensor readings are transfered over a low power bluetooth 
adapter to the computer (Fig. 1). 



Figure 1: MYO Armband Kit with Bluetooth Dongle 

The Software Development Kit (SDK) allow the developers to access the 
EMG signals and motion parameters on the worn arm. As the measured 
EMG signals depend on the sensor location on the muscles, the MYO arm- 
band applications require a special calibration gesture must be performed 
every time when the armband is put on the arm or taken off. In this way, 
the application calibrate the sensor locations depending on the sensed sensor 
direction and rotation of the armband. In our experiments we collected the 
experimental data without giving a break and taking off without needing 
re-calibrating the device. 

The IMU unit reports the motion related parameters at a frequency of 
50 Hz, whereas the frequency for the EMG signal is 200 Hz as reported by 
the company website na. The SDK provides a gesture object by which Eve 
different hand gestures are reported. The same hand gesture set which is 
composed of Hand Fist, Wave In, Wave Out, Hand Spread and Double Tap 
is used to test the SGRG performance (Fig. [^. If none of them is performed, 
the hand is assigned to the hand relax gesture. 

The location of the EMG sensors are important for certain types of ges- 
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Figure 2: MYO Hand Gestures, 1- Fist, 2- Hand Relax, 3- Finger Spread, 4- Wave In, 5- 
Wave Out, 6- Double Tap 

tures as different hand and finger movements are controlled by the different 
muscle groups on the forearm. The armband conhguration are put on close 
to the elbow where the discriminant EMG signals can be captured for hand 
movements. In order to classify individual huger gestures, the EMG sensors 
must be placed on the related muscle groups for huger resolution. 

3.2. Training 

In gesture recognition studies one of the open problems is spotting the 
gestures on a streaming signals. A gesture has a start and end points which 
can be spotted on a continuously monitored signal, however gestures might 
diher among the people even for the same performer. Along with the in- 
tra and inter personal variations, the time length of the diherent gestures 
in a set might also show variations. The spotting problem manifests itself 
in building a training dictionary for the representation based classihcation 
methods. This problem can be shortcut using a switch which is used to 
mark the beginning and end points of the gestures in real world and a trigger 
in the software applications. In the study, the authors collect gestures by 
drawing in the air using an IR pen which is tracked by the IR cameras of a 
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stereo vision Wiimote experimental setup. The performer press and release 
the switch of the IR pen at the beginning and the end of the gestures respec¬ 
tively. Spotting can also be done visually by analyzing the trajectory or the 
visually discriminating features which requires tedious effort for fairly rich 
gesture sets. 

In this study, as the EMG signals are highly jittery and the gesture pat¬ 
terns cannot be spotted visually on the figures of the signal we employ sub¬ 
space clustering methods for the repeated couple gestures. We used this 
approach in our previous studies. The reason is twofold to adapt this ap¬ 
proach. The hrst is the drawback of the constant sliding window which is 
also important for the CRC method. Both in the real-time applications and 
the training phase for building a representative dictionary, the signals are 
captured by a sliding window. The second is Ending representatives for ev¬ 
ery gesture state in any time window which afiect the recognition accuracy. 
As the tracked body part, in our case the hand might switch between ges¬ 
tures while being tracked, in the switching regions there occur overlapping 
gestures in the monitoring time window which degenerates the robustness 
and accuracy of the recognition. 

We overcome these two drawbacks employ a subspace method for a re¬ 
peated gesture couple. At least two gestures are necessary to cluster them 
into respective classes for the subspace clustering algorithms, the state of 
art exploits the self similarity property of the signals to be clustered. The 
most eminent of these algorithms are the Sparse and Low Rank Subspace 
Clustering (SSC, LRR) methods which make use of and nuclear norm in 
their objective functions |2Hl|29]. We use a variant of SSC, the Ordered Sub¬ 
space Clustering (CSC) which puts an additional penalty in the self similarity 
objective function aiming the sequential data [HO]- The additional term in 
the objective function enforces the neighboring representative samples to be 
same by penalizing the difierences of two neighbors. 

We collected gesture couples for clustering the obtained signal into two 
difierent classes using the OSC. The hand performs two gestures repeatedly. 
As the hand the hand relax state (which is actually not a gesture) every 
gestures ends with hand relax state. This state is assumed to be gesture 
mate of the each gesture. As a concrete explanation, let’s assume that a hand 
performs the hand wave in gesture and returns to the hand relax position 
repeatedly as in demonstrated in Fig. 
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Figure 3: Wave In - Hand Relax - Wave In Repetition 

On the recorded EMG signal, a constant length sliding window with an 
increment of sampling interval captnre the content which is then pnt into 
the combined dictionary matrix. The snbspace clustering method is used in 
this step to distill the pure representatives into their respective classes. A 
sample clustering result is given in Fig. for the double tap gesture on a 
single channel EMG signals. As seen, the period of the one gesture cycle is 
approximately 0.5 seconds corresponding to 100 time samples. 
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Figure 4: Clustered Double Tap - Hand Relax Gestures on EMG data 

In fact, although it is not seen on the figure, there are four hand states in 
repetitions of the gesture couples; two static hand postures where the hand 
waits at the hand relax and terminal waved in positions, and the dynamics 
hand gestures through these postures. However, as the number of subspaces 
are chosen as two for the gesture couples, the posture states are embedded 
into the clusters automatically and every observable hand states are repre¬ 
sented in these clusters. The training dictionary contains 10 gesture classes 
in this case and we map every return states to the hand relax position from 
any hand gesture are mapped to the five hand gestures. 

4. Simulation Results 

We collected five gesture couple experiments, one for training and the rest 
for testing. In addition, we perform an arbitrary hand gesture sequence in 
which the hand performs arbitrarily performs gestures arbitrarily in the se¬ 
quence. We also repeated the recognition simulations using the conventional 
CRC_RLS and compared the results. 

Although, the conventional CRC_RLS yields the same recognition accu¬ 
racy for some of the experimental data, it can not discriminate the hand 
wave in and fist gestures labeling the wave in as fist in all the experiments 
(Fig. § with some number of misclassification with the hand spread gesture. 
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However, the worst recognition result for the hand wave in gestures among 
the four test sets is 99.36% with the SCRC method. 
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Figure 5: Wave In - Hand Relax Gestures on EMG data with GRC 

Fig. § shows the worst result of the wave in - hand relax test experi¬ 
ment. The bottom hgure is the gesture label of the MYO gesture recognition 
module. Since the SRC recognizes 10 gestures the number of labels are dif¬ 
ferent then the MYO gesture module which labels the gestures as 1- Hand 
Relax, 2- Fist, 3- Wave In, 4- Wave Out, 5- Hand Spread and 6- Double Tap. 
The number of labeling computation in the hgure is 1219 and the SCRC 
mis-classihes the signal patterns as the hst gesture eight times. 
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Figure 6: Wave In - Hand Relax Gestures on EMG data with SGRC 


The recognition accnracy the hst and the double-tap gestures is 100% for 
all the test experiments (Figs. 
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Figure 7: Recognition Results for Fist and Hand Relax Experiment, Spectral CRC and 
MYO 
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Figure 8: Recognition Results for Double Tap and Hand Relax, Spectral CRC and MYO 


The results of the hngers spread recognition is given in Fig. The worst 
recognition accuracy in the study among all the experiment is obtained for 
this hand gesture as 97.3%. The algorithm gives some mis-classihcations 
for huger spreads and the wave out couple as there are overlapping wrist 
contraction and hexion states for the two hand movements that the hand 
might visit involuntarily. 
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Figure 9: Recognition Results for Hand Spread and Hand Relax, Spectral CRC and MYO 


The hand wave out gesture recognition results are given in Fig. [10] without 
mapping the hand relax return gesture to the hand relax state. In all the 
hgures the mapped hand relax state is labeled as one. In the following hgure 
the label six show that the hand is moving toward to the hand relax position. 
The worst recognition accuracy out of four test experiments is 98.34% for 
1274 labeling computation in the wave out experiment. The results when 


the CRC_RLS is given in Fig. 11, in which the accuracy is less then 85%. 
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Figure 10: Recognition Results for Wave Out and Hand Relax, Spectral CRC and MYO 



Figure 11: Recognition Results for Wave Out with CRC.RLS 
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As the final experiment we give the results of an arbitrary gesture se¬ 
quence. In this experiment, unlike the other experiments where the hand 
only performs two hand gestures, different gestures are performed arbitrarily 
(Fig. 12). The recognition accuracy is 98.47% for this experiment. 
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Figure 12: Recognition Results for Random Hand Gesture Sequence, Spectral CRC and 
MYO, 1- Hand Relax, 2- Fist, 4- Wave In, 6- Wave Out, 8- Hand Spread 


5. Conclusion and Future Works 

In this study we introduced a novel application of the CRC methods in 
Fourier domain and elaborated the methodologies and experimental results 
for the hand gesture recognition using the EMC measurements on a forearm. 
The accuracy levels for the fairly rich gesture set is promising for the raw 
EMC signals, therefore, they can be further improved by analyzing the signal 
noise and introducing the appropriate filtering methods. The use of Fourier 
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features is explained intuitively using the circulant approach which reveals 
shift invariant features in an elegant way. The approach detailed in the 
training phase which we used for detecting braking maneuvers of a mobility 
robot hrst time [3T] leads to a dictionary with a high power of discrimination. 
The embedding of the gesture boundaries into the representative states are 
implicitly realized by the subspace clustering methods. 

The methods and procedures we detailed in this paper is a small portion 
of a project by which a multi-modal intuitive HCI have been developed for 
the elderly or severely handicapped people. The motivation of the project to 
enable these people to steer a robotic wheelchair without excessive cognitive 
or physical efforts. On the other hand the developed systems will be employed 
for augmented and virtual reality environments to establish training theaters 
for people who are prescribed power wheelchairs hrst time. 
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